Search for: All records

In this paper we propose a new framework—MoViLan (Modular Vision and Language) for execution of visually grounded natural language instructions for day to day indoor household tasks. While several data-driven, end-to-end learning frameworks have been proposed for targeted navigation tasks based on the vision and language modalities, performance on recent benchmark data sets revealed the gap in developing comprehensive techniques for long horizon, compositional tasks (involving manipulation and navigation) with diverse object categories, realistic instructions and visual scenarios with non reversible state changes. We propose a modular approach to deal with the combined navigation and object interaction problem without the need for strictly aligned vision and language training data (e.g., in the form of expert demonstrated trajectories). Such an approach is a significant departure from the traditional end-to-end techniques in this space and allows for a more tractable training process with separate vision and language data sets. Specifically, we propose a novel geometry-aware mapping technique for cluttered indoor environments, and a language understanding model generalized for household instruction following. We demonstrate a significant increase in success rates for long horizon, compositional tasks over recent works on the recently released benchmark data set -ALFRED. 

Abstract Data-driven analysis and monitoring of complex dynamical systems have been gaining popularity due to various reasons like ubiquitous sensing and advanced computation capabilities. A key rationale is that such systems inherently have high dimensionality and feature complex subsystem interactions due to which majority of the first-principle based methods become insufficient. We explore the family of a recently proposed probabilistic graphical modeling technique, called spatiotemporal pattern network (STPN) in order to capture the Granger causal relationships among observations in a dynamical system. We also show that this technique can be used for anomaly detection and root-cause analysis for real-life dynamical systems. In this context, we introduce the notion of Granger-STPN (G-STPN) inspired by the notion of Granger causality and introduce a new nonparametric technique to detect causality among dynamical systems observations. We experimentally validate our framework for detecting anomalies and analyzing root causes in a robotic arm platform and obtain superior results compared to when other causality metrics were used in previous frameworks. 

Occupancy detection systems are commonly equipped with high quality cameras and a processor with high computational power to run detection algorithms. This paper presents a human occupancy detection system that uses battery-free cameras and a deep learning model implemented on a low-cost hub to detect human presence. Our low-resolution camera harvests energy from ambient light and transmits data to the hub using backscatter communication. We implement the state-of-the-art YOLOv5 network detection algorithm that offers high detection accuracy and fast inferencing speed on a Raspberry Pi 4 Model B. We achieve an inferencing speed of ∼100ms per image and an overall detection accuracy of >90% with only 2GB CPU RAM on the Raspberry Pi. In the experimental results, we also demonstrate that the detection is robust to noise, illuminance, occlusion, and angle of depression.